Method and system for clear signal capture

ABSTRACT

A method and system for clear signal capture comprehend several individual aspects that address specific problems in improved ways. In addition, the method and system also comprehend a hands-free implementation that is a practical solution to a very complex problem. Individual aspects comprehended related to echo and noise reduction, and divergence control.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. application Ser. No.10/914,898, filed Aug. 10, 2004.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to a method and system for capturing signals andto associated signal processing techniques. This invention furtherrelates to a method and system for hands free operation of mobile ornon-mobile phones.

2. Background Art

In any system for capturing signals, the goal is to capture the desiredsignal while rejecting undesired signals. Signal processing techniquesare employed to process a received input signal to enhance the desiredsignal while removing the undesired signals.

One particular problem faced in systems for hands free operation ofmobile or non-mobile phones is the acoustic echo cancellation (AEC)problem. The AEC problem is a well known problem, and it can bedescribed as shown in FIG. 1, where the far-end received signal (x(n))is sent to a loud speaker inside of a car (for example). This signal ispropagated by the interior of the automobile through the acoustic path(q(n)), and is fed back into the microphone generating the echo signal(c(n)). To cancel the echo signal an adaptive filter is used, where theobjective is to identify the acoustic echo path (q(n)) with the adaptivefilter (g(n)), and then to subtract the resultant signal (y(n)) from themicrophone signal. If (g(n)=q(n)) then (y(n)=c(n)), and the subtractionof the output signal of the adaptive filter from the microphone signalwill cancel the echo signal.

This AEC problem has been addressed in existing applications by usingdifferent types of adaptive filter algorithms such as least mean squarealgorithm (LMS), normalized least mean square algorithm (NLMS), datareuse normalized least mean square algorithm (DRNLMS), recursive leastsquare algorithm (RLS), affine projection algorithm (APA), and others.

Another related problem is that an adaptive filter algorithm needs sometype of control to prevent the divergence of the algorithm when far-endsend and near-end receive signals are present at the same time.

This divergence problem has been addressed in existing applications byintroducing a double talk detector (DTD). The DTD restricts theconditions under which the adaptive filter algorithm may adapt.

One particular requirement of any system is that the system must performwell in the presence of a noise signal (v(n)). In attempts to meet thisrequirement, a noise cancellation algorithm (NC) has been introduced.Various different approaches have been taken for implementing the NCalgorithm including approaches based on spectral subtraction, Kalmanfilters, neural networks, and others.

In another aspect, existing applications have introduced a non-linearprocessor (NLP). The NLP attempts to compensate for the practicalproblem of the adaptive filter algorithm not achieving its minimum meansquare error (MSE) and for system non-linearity particularly where oneof the sources is the non-linear loud speaker.

Overall, existing applications have taken a variety of approaches toaddress acoustic echo, adaptive algorithm divergence, noise, and systemnon-linearity. The initial problem of acoustic echo cancellation hasdeveloped into an evolving complex problem involving a number ofdifferent design aspects. Although various approaches have been taken inaddressing specific issues, the overall evolving complex problem has yetto be fully addressed.

Background information may be found in S. Haykin, Adaptive FilterTheory, Prentice Hall, Upper Saddle River, N.J., 4th Edition, 2002; P.S. R. Diniz, Adaptive Filtering—Algorithms and Practical Implementation,Kluwer Academic Publishers, Dordrecht, The Netherlands, 2nd Edition,2002; P. P. Vaidyanathan, Multirate Systems and Filter Banks, PrenticeHall Signal Processing Series, Englewood Cliffs, N.J., 1993; R. E.Crochiere, L. R. Rabiner, Multirate Digital Signal Processing, PrenticeHall, Englewood Cliffs, N.J.; S. T. Gay, J. Benesty, Acoustic SignalProcessing for Telecommunication, Kluwer Academic Publishers, Dordrecht,The Netherlands, 2000; S. F. Boll, “Suppression of acoustic noise inspeech using spectral subtraction,” IEEE Trans. Acoust., Speech, SignalProc., vol. ASSP-27, April 1979; R. B. Jeannes, P. Scalart, G. Faucon,C. Beaugeant, “Combined noise and echo reduction in hands free systems:A survey,” IEEE Trans. Speech Audio Processing, vol. 9, pp 808-820,November 2001; R. Martin, J. Altenhoner, “Coupled Adaptive Filters forAcoustic Echo Control and Noise Reduction,” Proc. ICASSP 95, pp.3043-3046, May 1995; M. R. Petraglia, R. G. Alves, P. S. R. Diniz, “NewStructures for Adaptive Filtering in Subbands with Critical Sampling,”IEEE Transactions on Signal Processing, Vol. 48, No. 12, December 2000;M. R. Petraglia, R. G. Alves, P. S. R. Diniz, “Convergence Analysis ofan Oversampled Subband Adaptive Filtering Structure with Local Errors,”Proc. IEEE Int. Symp. on Circuits and Systems (ISCAS), May 2000.

For the foregoing reasons, there is a need for an improved method andsystem for clear signal capture that provides a practical solution tothis evolving complex problem.

SUMMARY OF THE INVENTION

It is an object of the invention to provide an improved method andsystem for clear signal capture. The improved method and systemcomprehend several individual aspects that address specific problems inimproved ways. In addition, the improved method and system alsocomprehend a hands free implementation that is a practical solution to avery complex problem.

In carrying out the invention, a method and system for clear signalcapture are provided. The method and system comprehend severalindividual aspects that address specific problems in improved ways.

In one aspect of the invention, an improved technique is used toimplement acoustic echo cancellation (AEC) and noise cancellation (NC).This aspect involves using a frequency domain approach for both AEC andNC. Preferably, the input microphone signal and the speaker signal aresplit into subbands for independent processing.

More specifically, a method of acoustic echo cancellation (AEC) andnoise cancellation (NC) is provided. A microphone signal resulting froman unobservable signal corrupted by additive background noise (thenear-end component) and an acoustic echo (the far-end component which isthe speaker signal modified by the acoustic path) is processed in anattempt to restore the unobservable signal. At a more detailed level,the original microphone signal in the time domain is processed by ananalysis filter bank to result in a frequency domain representation ofthe microphone signal. The speaker signal is also processed by ananalysis filter bank to result in a frequency domain representation ofthe speaker signal. The (frequency domain) speaker signal is processedby an adaptive filter that models the echo path. The (frequency domain)microphone signal is processed by a noise cancellation filter. Theoutput of the adaptive filter is processed by a copy of the noisecancellation filter. The outputs of the noise cancellation filter andfilter copy are compared using subtraction to determine an error, andthe adaptive filter that models the echo path is adapted based on theerror.

This approach allows the converging adaptive filter to have the benefitof noise cancelling before comparing so that the adaptive filter canbetter model the echo path. A second noise cancellation filter isapplied to an echoless signal that is obtained by directly comparing theadaptive filter output to the microphone signal. In this way, theadaptive filter tracking benefits from the first noise cancellationfilter and its copy, and the second noise cancellation filter is appliedto an echoless signal obtained via direct comparison to provide theestimation of the unobservable signal.

In one aspect of the invention, an improved technique is used toimplement noise cancellation. A method of frequency domain-based noisecancellation is provided. A noisy signal resulting from an unobservablesignal corrupted by additive background noise is processed in an attemptto restore the unobservable signal. The method comprises estimatingbackground noise power with a recursive noise power estimator having anadaptive time constant, and applying a filter based on the backgroundnoise power estimate in an attempt to restore the unobservable signal.

Preferably, the background noise power estimation technique considersthe likelihood that there is no speech power in the current frame andadjusts the time constant accordingly. In this way, the noise powerestimate tracks at a lesser rate when the likelihood that there is nospeech power in the current frame is lower. In any case, sincebackground noise is a random process, its exact power at any given timefluctuates around its average power.

To avoid musical or watery noise that would occur due to the randomnessof the noise particularly when the filter gain is small, the methodfurther comprises smoothing the variations in a preliminary filter gainto result in an applied filter gain having a regulated variation.Preferably, an approach is taken that normalizes variation in theapplied filter gain. To achieve an ideal situation, the average rateshould be proportional to the square of the gain. This will reduce theoccurrence of musical or watery noise and will avoid ambiance. In oneapproach, a pre-estimate of the applied filter gain is the basis fornormalizing the adaption rate.

In one aspect of the invention, an improved technique is used to addressthe divergence of the adaptive filter algorithm. A method of acousticecho cancellation (AEC) is provided. A microphone signal resulting froman unobservable signal corrupted by additive background noise (thenear-end component) and an acoustic echo (the far-end component which isthe speaker signal modified by the acoustic path) is processed in anattempt to restore the unobservable signal. The speaker signal isprocessed by an adaptive filter that models the echo path. The output ofthe adaptive filter and the microphone signal are compared bysubtraction to determine an error, and the adaptive filter that modelsthe echo path is adapted based on the error to allow the adaptive filterto converge.

The approach of this method allows the converging adaptive filter toachieve an optimal balance between dynamic tracking ability and filterdivergence control. In addition, gain control may be independentlyperformed in each subband.

The method comprises controlling the adaption gain of the adaptivefilter based on the ratio of far-end energy to total energy. Thisrequires an approximation of the far-end component (or equivalently thenear-end component). The result is that when far-end echo dominates,tracking occurs quickly to achieve dynamic tracking, and when near-endspeech or noise dominates, tracking occurs more slowly to achievedivergence control.

At a more detailed level, a preferred approach to far-end energyapproximation considers the correlation between the microphone signaland the filter output signal. The preferred adaptation gain is based ona ratio of this correlation to the expected total microphone signalenergy. In more detail, the adaptation gain should be proportional tothe square of this ratio.

In a most preferred approach, the adaptation gain control methodincludes further arrangements for handling exception conditions that maybe present in certain applications. In these exceptions, what happens isthat the far-end energy is underestimated. This could happen at systemreset or when the echo path gain suddenly increases. Exceptionconditions are handled by assuring that the adaption rate is not overlydiminished due to the resultant underestimation of the energy ratio.

In another aspect of the invention, a method of residual echo reduction(RERF) is provided. This method is performed after the initial acousticecho cancellation is performed. The target signal to the residual echoreduction filter is the main AEC filter output. The RERF is also anadaptive filter algorithm and the RERF input signal is the system inputsignal or alternatively a function thereof that hastens convergence ofthe RERF adaptation.

In this way, the RERF gain effectively converges toward zero when thesystem input signal is mostly far-end echo causing the AEC output toconverge toward zero. On the other hand, when the microphone input ismostly speech, the AEC output reflects the system input and the RERFeffectively approaches unity (in the case where RERF input is the sameas the microphone input). The resultant gain of the RERF is then applieddirectly to the AEC output signal to reduce residual echo present at theAEC output signal.

At a more detailed level, this aspect of the invention comprehendscontrolling the RERF adaptation rate. The method comprehends a RERFadaptation rate step size control applied to the filter gain constant.The step size control is based on a weighted average of the previousgain and a look ahead. The weighting of the look ahead increases asfrequency increases and provides greater flexibility at higherfrequencies.

In yet another aspect of the invention, a divergence control methodprotects the output of the system from rare divergence of the adaptivealgorithm. The method is based on conservation of energy.

The method may be used at any of the adaptive filters. At a moredetailed level, the method involves comparing the energy of the filteroutput signal to the energy of the target signal. The filter outputsignal should have less energy than the target signal. In the event thatit is concluded based on the energy comparison that the filter hasdiverged, the target signal bypasses the error signal and becomes thestage output.

In another aspect of the invention, a hands-free implementation mayinvolve any combinations of the various individual aspects that addressspecific problems. But it is to be appreciated that these individualaspects each are useful alone, as well as in combinations.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a prior art hands free implementation;

FIG. 2 illustrates a subband adaptive filter structure;

FIG. 3 illustrates a noise cancellation algorithm;

FIG. 4 illustrates an arrangement that optimizes the AEC algorithmperformance;

FIG. 5 illustrates an arrangement that optimizes the AEC and NCalgorithms performance;

FIG. 6 illustrates main and auxiliary adaptive filters in an arrangementthat overcomes deficiencies of a traditional DTD arrangement;

FIG. 7 illustrates a residual echo reduction filter;

FIG. 8 illustrates a divergence control system;

FIGS. 9A-9B illustrate a non-linear processor implementation; and

FIG. 10 illustrates a method and system for a clear signal capture inthe preferred embodiment which incorporates preferred embodiments of theimproved individual features.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 2 illustrates a subband adaptive filter structure. The adaptivefilter algorithm used is the second-order DRNLMS in the frequencydomain. The noise cancellation algorithm is illustrated in FIG. 3, andis also implemented in the frequency domain. In this way, both thespeaker and microphone signals are split into frequency subbands, theAEC and NC are implemented in frequency domain, and the output signal istransformed back to the time domain.

With continuing reference to FIG. 2, the subband adaptive filterstructure used to implement the NLMS in subbands consists of twoanalysis filter banks, which split the speaker (x(n)) and microphone(d(n)) signals into M bands each. The subband signals are modified by anadaptive filter, after being decimated by a factor L, and thecoefficients of each subfilter (G_(i)) are adapted independently usingthe individual error signal of the corresponding band (E_(i)). In orderto avoid aliasing effects, this structure uses a down-sampling factor Lsmaller than the number of subbands M. The analysis and synthesis filterbanks can be implemented by uniform DFT filter banks, so that theanalysis and synthesis filters are shifted versions of the low-passprototype filters, i.e.H _(i)(z)=H ₀(zW _(M) ^(i))F _(i)(z)=F ₀(zW _(M) ^(i))with i=0, 1, . . . , M−1, where H₀(z) and F₀(z) are the analysis andsynthesis prototype filters, respectively, and

$W_{M} = {{\mathbb{e}}^{{- j}\;\frac{2\pi}{M}}.}$Uniform filter banks can be efficiently implemented by the WeightedOverlap-Add (WOA) method.

The coefficient update equation for the subband structure of FIG. 2,based on the NLMS algorithm, is given by:G _(i)(k+1)= G _(i)(k)+μ_(i)(k)[ X _(i)*(k)E _(i)(k)]where ‘*’ represents the conjugate value of X _(i)(k), and:

E_(i)(k) = D_(i)(k) − Y_(i)(k)${Y_{i}(k)} = {{{\underset{\_}{X}}_{i}^{T}(k)}{{\underset{\_}{G}}_{i}(k)}}$${\mu_{i}(k)} = \frac{\mu}{P_{i}(k)}$are the error signal, the output of the adaptive filter and thestep-size in each subband, respectively.

Note that the step size appears normalized by the power of the referencesignal. Note also that μ is a constant real value, and P_(i)(k) is thepower estimate of the reference signal X_(i)(k), which can be obtainedrecursively by the equation:P _(i)(k+1)=βP _(i)(k)+(1−β)|X _(i)(k)|²for 0<β<1.

A few observations of the illustrated subband adaptive filter can bemade. If the system to be identified has N coefficients in fullband,each subband adaptive filter (G _(i)(k)) will be a column vector withN/L complex coefficients, as well as X _(i)(k). D_(i)(k), X_(i)(k),Y_(i)(k) and E_(i)(k) are complex numbers. The choice of N is related tothe tail length of the echo signal to cancel, for example, if fs=8 kHz,and the desired tail length is 64 ms, N=8000*0.064=512 coefficients, forthe time domain fullband adaptive filter. β is related to the number ofcoefficients of the adaptive filter ((N−L)/N). The number of subbandsfor real input signals is M=(Number of FFT points)/2+1.

The previous equations describe the NLMS in subband, to obtain theDRNLMS it is required to compute the “new” error signal (E_(i)(k)) usingthe updated values of the subband adaptive filter coefficients, and toupdate again the coefficients of the subband adaptive filters, it is:

${Y_{i}^{j}(k)} = {{{\underset{\_}{X}}_{i}^{T}(k)}{{\underset{\_}{G}}_{i}^{j - 1}(k)}}$E_(i)^(j)(k) = D_(i)(k) − Y_(i)^(j)(k)${\mu_{i}^{j}(k)} = \frac{\mu^{j}}{P_{i}(k)}$${{\underset{\_}{G}}_{i}^{j}(k)} = {{{\underset{\_}{G}}_{i}^{j - 1}(k)} + {{\mu_{i}^{j}(k)}\left\lbrack {{{\underset{\_}{X}}_{i}^{*}(k)}{E_{i}^{j}(k)}} \right\rbrack}}$where j=2, . . . R represents the number of reuses that are in thealgorithm, also known as order of the algorithm. Observe thatG _(i) ¹(k)= G _(i)(k)μ_(i) ¹(k)=μ_(i)(k)E _(i) ¹(k)=E _(i)(k)andY _(i) ¹(k)=Y _(i)(k)

With continuing reference to FIG. 3, the noise cancellation algorithmconsiders that a speech signal s(n) is corrupted by additive backgroundnoise v(n), so the resulting noisy speech signal d(n) can be expressedasd(n)=s(n)+v(n).

Ideally, the goal of the noise cancellation algorithm is to restore theunobservable s(n) based on d(n). Unlike the AEC problem, where whatneeds to be removed from the microphone signal is unambiguous, the noisecancellation problem is usually not as well defined. For the purpose ofthis noise cancellation algorithm, the background noise is defined asthe quasi-stationary noise that varies at a much slower rate compared tothe speech signal.

The noise cancellation algorithm is a frequency-domain based algorithm.With a DFT analysis filter bank with length (2M−2) DFT, the noisy signald(n) is split into M subband signals, D_(i)(k), i=0, 1 . . . , M−1, withthe center frequencies uniformly spaced from DC to Nyquist frequency.Except the DC and the Nyquist bands (bands 0 and M−1, respectively), allother subbands have equal bandwidth which equals to 1/(M−1) of theoverall effective bandwidth. In each subband, the average power ofquasi-stationary background noise is tracked, and then a gain is decidedaccordingly and applied to the subband signals. The modified subbandsignals are subsequently combined by a DFT synthesis filter bank togenerate the output signal. When combined with other frequency-domainmodules (AEC for example), the DFT analysis and synthesis banks aremoved to the front and back of all modules, respectively.

Because it is assumed that the background noise varies slowly comparedto the speech signal, its power in each subband can be tracked by arecursive estimator

$\begin{matrix}{{P_{{NZ},i}(k)} = {{\left( {1 - \alpha_{NZ}} \right){P_{{NZ},i}\left( {k - 1} \right)}} + {\alpha_{NZ}{{D_{i}(k)}}^{2}}}} \\{= {{P_{{NZ},i}\left( {k - 1} \right)} + {\alpha_{NZ}\left( {{{D_{i}(k)}}^{2} - {P_{{NZ},i}\left( {k - 1} \right)}} \right)}}}\end{matrix}$where the parameter α_(NZ) is a constant between 0 and 1 that decidesthe weight of each frame, and hence the effective average time. Theproblem with this estimation is that it also includes the power ofspeech signal in the average. If the speech is not sporadic, significantover-estimation can result. To avoid this problem, a probability modelof the background noise power is used to evaluate the likelihood thatthe current frame has no speech power in the subband. When thelikelihood is low, the time constant α_(NZ) is reduced to drop theinfluence of the current frame in the power estimate. The likelihood iscomputed based on the current input power and the latest noise powerestimate:

${L_{{NZ},i}(k)} = {\frac{{{D_{i}(k)}}^{2}}{P_{{NZ},i}\left( {k - 1} \right)}{\exp\left( {1 - \frac{{{D_{i}(k)}}^{2}}{P_{{NZ},i}\left( {k - 1} \right)}} \right)}}$and the noise power is estimated asP _(NZ,i)(k)=P _(NZ,i)(k−1)+(α_(NZ) L _(NZ,i)(k))(|D _(i)(k)|² −P_(NZ,i)(k−1)).

It can be observed that L_(NZ,i)(k) is between 0 and 1. It reaches 1only when |D_(i)(k)|² is equal to P_(NZ,i)(k−1), and reduces towards 0when they become more different. This allows smooth transitions to betracked but prevents any dramatic variation from affecting the noiseestimate.

In practice, less constrained estimates are computed to serve as theupper- and lower-bounds of P_(NZ,i)(k). When it is detected thatP_(NZ,i)(k) is no longer within the region defined by the bounds, it isadjusted according to these bounds and the adaptation continues. Thisenhances the ability of the algorithm to accommodate occasional suddennoise floor changes, or to prevent the noise power estimate from beingtrapped due to inconsistent audio input stream.

In general, it can be assumed that the speech signal and the backgroundnoise are independent, and thus the power of the microphone signal isequal to the power of the speech signal plus the power of backgroundnoise in each subband. The power of the microphone signal can becomputed as |D_(i)(k)|². With the noise power available, an estimate ofthe speech power isP _(SP,i)(k)=max(|D _(i)(k)|² −P _(NZ,i)(k),0)and therefore, the optimal Wiener filter gain can be computed as

${G_{T,i}(k)} = {{\max\left( {{1 - \frac{P_{{NZ},i}(k)}{{{D_{i}(k)}}^{2}}},0} \right)}.}$

However, since the background noise is a random process, its exact powerat any given time fluctuates around its average power even if it isstationary. By simply removing the average noise power, a noise floorwith quick variations is generated, which is often referred to asmusical noise or watery noise. This is the major problem with algorithmsbased on spectral subtraction. Therefore, the instantaneous gainG_(T,i)(k) needs to be further processed before being applied.

When |D_(i)(k)|² is much larger than P_(NZ,i)(k), the fluctuation ofnoise power is minor compared to |D_(i)(k)|², and hence G_(T,i)(k) isvery reliable. On the other hand, when |D_(i)(k)|² approximatesP_(NZ,i)(k), the fluctuation of noise power becomes significant, andhence G_(T,i)(k) varies quickly and is unreliable. In accordance with anaspect of the invention, more averaging is necessary in this case toimprove the reliability of gain factor. To achieve the same normalizedvariation for the gain factor, the average rate needs to be proportionalto the square of the gain. Therefore the gain factor G_(oms,i)(k) iscomputed by smoothing G_(T,i)(k) with the following algorithm:G _(oms,i)(k)=G _(oms,i)(k−1)+(α_(G) G _(0,i) ²(k)(G _(T,i)(k)−G_(oms,i)(k−1))G _(0,i)(k)=G _(oms,i)(k−1)+0.25×(G _(T,i)(k)−G _(oms,i)(k−1))where α_(G) is a time constant between 0 and 1, and G_(0,i)(k) is apre-estimate of G_(oms,i)(k) based on the latest gain estimate and theinstantaneous gain. The output signal can be computed asŜ _(i)(k)=G _(oms,i)(k)×D _(i)(k).

It can be observed that G_(oms,i)(k) is averaged over a long time whenit is close to 0, but is averaged over a shorter time when itapproximates 1. This creates a smooth noise floor while avoidinggenerating ambient speech.

FIG. 4 illustrates the subband adaptive filter structure and the noisecancellation algorithm arranged to optimize AEC performance.

Considering the prior art system shown in FIG. 1, the adaptive filteralgorithm comes first and the noise cancellation implementation follows.It is realized in this way because historically the adaptive filteralgorithm is realized in time domain, and the noise cancellationalgorithm is realized in frequency domain. Changing the order of theadaptive filter and the noise cancellation algorithms would introduce adelay at the microphone signal path caused by the NC algorithm and wouldalso introduce a non-linearity caused by the NC algorithm. The adaptivefilter cannot compensate non-linearity (because it is a linear system(FIR filter)).

The arrangement of FIG. 4 overcomes these limitations by implementingthe NC algorithm first and then the adaptable filter algorithm second.In this way, environmental noise is removed and the convergence rate ofthe adaptive filter algorithm and also the maximum echo return lossenhancement (ERLE) obtained by the system will be increased.

By placing a copy of the OMS gain coefficients at the output of thesubband adaptive filters, the non-linearity introduced by the OMS gaincoefficients at the microphone input is compensated. In addition, byimplementing the subband adaptive filter structure and noisecancellation algorithm in the frequency domain, time delay problems canbe avoided.

FIG. 5 illustrates the subband adaptive filter structure and the noisecancellation algorithm arranged to optimize AEC and NC performance. FIG.4 illustrates the AEC improvement achieved by placing the NC algorithmbefore the AEC algorithm, but from the NC algorithm point of view, theNC algorithm could perform better if applied after removal of the echo.The FIG. 5 arrangement optimizes both algorithms at the same time. Asshown, a second NC algorithm (OMS₂ block) is performed in an echolesssignal. That is, the adaptive filter benefits from the presence of theOMS₁ and OMS₁ copy blocks while the OMS₂ block benefits from theadaptive filter. In this way, the output from the OMS₂ block benefitssignificantly in terms of AEC and NC performance.

From the acoustic model illustrated in FIG. 5, the microphone signald(n) can be decomposed asd(n)=d _(ne)(n)+d _(fe)(n)where the near-end component d_(ne)(n) is the sum of the near-end speechs(n) and background noise v(n), and the far-end component d_(fe)(n) isthe acoustic echo, which is the speaker signal modified by the acousticpath: c(n)=q(n){circle around (×)}x(n). The NLMS filter estimates theacoustic path by matching the speaker signal (x(n)) to the microphonesignal (d(n)) through correlation. Because it is assumed that bothnear-end speech and background noise are uncorrelated to the referencesignal, the adaptive filter should converge to the acoustic path q(n).

However, since the NLMS is a gradient-based adaptive algorithm thatapproximates the actual gradients by single samples, the filtercoefficients drift around the ideal solutions even after the filterconverges. The range of drifting, or misadjustment, depends mainly ontwo factors: adaptation gain constant μ and the energy ratio betweennear-end and far-end components.

The misadjustment plays an important role in AEC performance. Whennear-end speech or background noise is present, this increases thenear-end to far-end ratio, and hence increases the misadjustment. Thusthe filter coefficients drift further away from the ideal solution, andthe residual echo becomes louder as a result. This problem is usuallyreferred to as divergence.

Traditional AEC algorithms deal with the divergence problem by deployinga state machine that categorizes the current event into one of fourcategories: silence (neither far-end nor near-end speech present),receive-only (only far-end speech present), send-only (only near-endspeech present), and double-talk (both far-end and near-end speechpresent). By adapting filter coefficients during the receive-only stateand halting adaptation otherwise, the traditional AEC algorithm preventsdivergence due to the increase in near-end to far-end ratio. Because thestate machine is based on the detection of voice activities at bothends, this method is often referred to as double-talk detection (DTD).

Although working nicely in many applications, the DTD inherits twofundamental problems. First of all, it completely ignores the near-endbackground noise as a factor. Secondly, it only allows filter adaptationin the receive-only state, and thus cannot handle any echo pathvariation during other states. The DTD can get away with these problemswhen the background noise level is ignorable and the near-end speech issporadic. However, when background noise becomes significant, not onlythe accuracy of state detection suffers, the balance between dynamictracking and divergence prevention also becomes difficult. Therefore, alot of tuning effort is necessary for a traditional DTD-based system,and system robustness is often a problem.

Furthermore, the traditional DTD-based system often manipulates theoutput signal according to the detected state in order to achieve betterecho reduction. This often results in half-duplex like performance innoisy conditions.

To overcome the deficiency of the traditional DTD, a more sophisticateddouble-talk control is used in order to achieve better overall AECperformance. As discussed above, the misadjustment mainly depends on twofactors: adaptation gain constant and near-end to far-end ratio.Therefore, using adaptation gain constant as a counter-balance to thenear-end to far-end ratio can keep the misadjustment at a constant leveland thus reduce divergence. To achieve this, it is necessary that

${\mu \propto \left( \frac{{far} - {{end}\mspace{14mu}{energy}}}{{total}\mspace{14mu}{energy}} \right)^{2}} = {\left( \frac{E\left\{ {{d_{fe}(n)}}^{2} \right\}}{E\left\{ {{d(n)}}^{2} \right\}} \right)^{2}.}$

When there is no near-end component, the filter adaptation is allowed toproceed at full speed. As the near-end to far-end ratio increases, thefilter adaptation slows down accordingly. Finally, when there is nofar-end component, the filter adaptation is halted since there is noinformation about the echo path available. Theoretically, this strategyachieves optimal balance between dynamic tracking ability and filterdivergence control. Furthermore, because the adaptive filter in eachsubband is independent from the filters in other subbands, this gaincontrol decision can be made independent in each subband and becomesmore efficient.

The major obstacle of this strategy is the availability of the far-end(or equivalently, near-end) component. With access to these components,there would be no need for an AEC system. Therefore, an approximate formis used in the adaptation gain control:

$\mu_{i}\frac{{{E\left\{ {{D_{i}(k)}{Y_{i}^{*}(k)}} \right\}}}^{2}}{E\left\{ {{D_{i}(k)}}^{2} \right\}^{2}}\gamma$where γ is a constant that represents the maximum adaptation gain. Whenthe filter is reasonably close to converging, Y_(i)(k) would approximatethe far-end component in the i-th subband, and therefore,E{D_(i)(k)Y*_(i)(k)} would approximate the far-end energy. In practice,it should be noted that the energy ratio should be limited to itstheoretical range bounded by 0 and 1 (inclusively). This gain controldecision works effectively in most conditions, with two exceptions whichwill be addressed in the subsequent discussion.

From the discussion above, E{D_(i)(k)Y*_(i)(k)} approximates the energyof the far-end component only when the adaptive filter converges. Thismeans that over- or under-estimation of the far-end energy can occurwhen the filter is far from convergence. However, increasedmisadjustment, or divergence, is a problem only after the filterconverges, so over-estimating the far-end energy actually helpsaccelerating the convergence process without causing a negativetrade-off. On the other hand, under-estimating the far-end energy slowsdown or even paralyzes the convergence process, and therefore is aconcern with the aforementioned gain control decision.

Specifically, under-estimation of far-end energy happens whenE{D_(i)(k)Y*_(i)(k)} is much smaller than the energy of far-endcomponent, E{|D_(fe,i)(k)|²}. By analyzing all possible scenarios,under-estimating mainly happens in the following two situations: (1)When the system is reset, with all filter coefficients initialized aszero, Y_(i)(k) would be zero. This leads to the adaptation gain μ beingzero and the adaptive system being trapped as a result. (2) When theecho path gain suddenly increases, the Y_(i)(k) computed based on theearlier samples would be much weaker than the actual far-end component.This can happen when the distance between speaker and microphone issuddenly reduced. Additionally, if the reference signal passes throughan independent volume controller before reaching the speaker, the volumecontrol gain would also figure into the echo path. Therefore turning upthe volume would also increase echo path gain drastically.

For the first situation, the adaptation gain control is suspended for ashort interval right after the system reset, which helps kick-start thefilter adaptation. For the second situation, an auxiliary filter(G′_(i)(k)) is introduced to relieve the under-estimation problem. Theauxiliary filter is a plain subband NLMS filter which is parallel to themain filter, as illustrated in FIG. 6, and with the number of taps beingenough to cover the main echo path. Its adaptation gain constant shouldbe small enough such that no significant divergence would result withoutany adaptation gain or double-talk control mechanism. After eachadaptation, the 2-norms of the main and auxiliary filters in eachsubband are computed:SqGa _(i)(k)=∥ G _(i)(k)∥₂SqGb _(i)(k)=∥ G′ _(i)(k)∥₂which are estimates of echo path gain from both filters, respectively.Since the auxiliary filter is not constrained by the gain controldecision, it is allowed to adapt freely all of the time. Theunder-estimation factor of the main filter can be estimated as

${RatSqG}_{i} = {\min\left( {\frac{{SqGa}_{i}(k)}{{SqGb}_{i}(k)},1} \right)}$and the double-talk based adaptation gain control decision can bemodified as

$\mu_{i} = {{\min\left( {\frac{{{E\left\{ {{D_{i}(k)}{Y_{i}^{*}(k)}} \right\}}}^{2}}{E\left\{ {{D_{i}(k)}}^{2} \right\}^{2} \times {RatSqG}_{i}},1} \right)}{\gamma.}}$

It can be observed that the auxiliary filter only affects systemperformance when its echo path gain surpasses that of the main filter.Furthermore, it only accelerates the adaptation of the main filterbecause RatSqG_(i) is limited between 0 and 1.

As discussed previously, the acoustic echo cancellation problem isapproached based on the assumption that the echo path can be modeled bya linear finite impulse response (FIR) system, which means that thefar-end component received by the microphone is the result of thespeaker signal transformed by an FIR filter. The AEC filter uses asubband NLMS-based adaptive algorithm to estimate the filter from thespeaker and microphone signals in order to remove the far-end componentfrom the microphone signal.

It can be observed that a residual echo is usually left in the output ofthe adaptive filter. Many factors can contribute to this. First of all,the linear FIR model might not be exactly satisfied. In addition, theecho path might be time-varying, which makes the adaptive filter lagbehind no matter how fast it adapts. Finally, the misadjustment of theadaptive filter results in filter estimation error even if the model isperfect, the echo path is time-invariant, and the algorithm is fullyconverged. This residual echo usually makes the AEC performanceunsatisfactory. Therefore, further attenuating the residual echo withoutsignificantly damaging the near-end speech is desired.

A residual echo reduction (RER) filter is illustrated in FIG. 7 and isused to achieve this goal. Following the overall system structure, itworks in each subband independently. For each subband, a one-tap NLMSfilter is implemented with the main AEC filter output (E_(i)(k)) as theideal signal. If the microphone signal (D_(i)(k)) is used as thereference signal, the one-tap filter will converge to

${G_{r,i}(k)} = {\frac{E\left\{ {{E_{i}(k)}{D_{i}^{*}(k)}} \right\}}{E\left\{ {{D_{i}(k)}}^{2} \right\}}.}$

When the microphone signal contains mostly far-end component, most of itshould be removed from E_(i)(k) by the main AEC filter and thus theabsolute value of G_(r,i)(k) should be close to 0. On the other hand,when the microphone signal contains mostly near-end component, E_(i)(k)should approximate D_(i)(k), and thus G_(r,i)(k) is close to 1.Therefore, by applying |G_(r,i)(k)| as a gain on E_(i)(k), the residualecho can be greatly attenuated while the near-end speech is mostlyintact.

To further protect the near-end speech, the input signal to the one-tapNLMS filter can be changed from D_(i)(k) to F_(i)(k), which is aweighted linear combination of D_(i)(k) and E_(i)(k) defined asF _(i)(k)=(1−R _(NE,i)(k))D _(i)(k)+R _(NE,i)(k)E _(i)(k)where R_(NE,i)(k) is an instantaneous estimate of the near-end energyratio. With this change, the solution of G_(r,i)(k) becomes

${G_{r,i}(k)} = {\frac{E\left\{ {{E_{i}(k)}{F_{i}^{*}(k)}} \right\}}{E\left\{ {{F_{i}(k)}}^{2} \right\}}.}$It can be observed that when R_(NE,i)(k) is close to 1, F_(i)(k) iseffectively E_(i)(k), and thus G_(r,i)(k) is forced to stay close to 1.On the other hand, when R_(NE,i)(k) is close to 0, F_(i)(k) becomesD_(i)(k), and G_(r,i)(k) returns to the previous definition. Therefore,the RER filter preserves the near-end speech better with thismodification while achieving similar residual echo reductionperformance.

Because |G_(r,i)(k)| is applied as the gain on E_(i)(k), the adaptationrate of the RER filter affects the quality of output signalsignificantly. If adaptation is too slow, the on-set near-end speechafter echo events can be seriously attenuated, and near-end speech canbecome ambient as well. On the other hand, if adaptation is too fast,unwanted residual echo can pop up and the background can become watery.To achieve optimal balance, an adaptation step-size control (ASC) isapplied to the adaptation gain constant of the RER filter:

μ_(r, i)(k) = ASC_(i)(k)γ_(r)${{ASC}_{i}(k)} = {{\left( {1 - \alpha_{{ASC},i}} \right){{G_{r,i}\left( {k - 1} \right)}}^{2}} + {\alpha_{{ASC},i}{{\min\left( {\frac{{{E_{i}(k)}}^{2}}{{{F_{i}(k)}}^{2}},1} \right)}.}}}$It can be observed that ASC_(i)(k) is decided by the latest estimate of|G_(r,i)|² plus a one-step look ahead. The frequency-dependent parameterα_(ASC,i), which decides the weight of the one-step look ahead, isdefined asα_(ASC,i)=1−exp(−M/(2i)),i=0, 1, . . . , (M/2)where M is the DFT size. This gives more weight to the one-steplook-ahead in the higher frequency subbands because the same number ofsamples cover more periods in the higher-frequency subbands, and hencethe one-step look-ahead there is more reliable. This arrangement resultsin more flexibility at higher-frequency, which helps preserve highfrequency components in the near-end speech.

The divergence control system basically protects the output of thesystem from rare divergence of the adaptive algorithm and it is based onthe conservation of energy theory for each subband of the hands freesystem.

FIG. 8 presents the block diagram of the divergence control system andit compares in each subband the power of the microphone signal(D_(i)(k)) with the power of the output of the adaptive filter(Y_(i)(k)). Because energy is being extracted (the echo) from themicrophone signal, the power of the adaptive filter output has to besmaller than or equal to the power of the microphone signal in eachsubband, if this does not happen it means that the adaptive subfilter isadding energy to the system and the assumption will be that the adaptivealgorithm diverged, if it occurs the output of the subtraction block(E_(i)(k)), is replaced by the microphone signal D_(i)(k)).

The divergence control system is also used for the subtraction blocksafter OMS1 and before the RER calculation to improve the performance ofthe RER in case of divergence of the main adaptive filter.

The objective of the comfort noise generator (CNG) is to compensate forthe irregularities at the noise floor in the output signal caused by theresidual echo reduction. Basically for each subband an estimate of themagnitude of the noise floor before the residual echo reduction block ismade, and compared with the magnitude of the signal after the residualecho reduction block. If the magnitude of the signal after the RER blockis smaller than the magnitude of the signal before the RER block, asignal with the magnitude of the result of the difference between thesetwo signals and random phase is added to the output signal, otherwisenothing is added. Observe that the CNG proposed also can compensateexisting discontinuities in frequency domain on the microphone signal,which will provide an audible improvement on the quality of the systemoutput signal.

The center-clipping also known as the non linear processor (NLP) isimplemented to remove some residual echo that is still present at theoutput signal, it works in time domain and it basically puts to zero thesamples that have absolute value smaller than a predefined threshold(Th). There are two different approaches, and they are presented inFIGS. 9A-9B.

The block diagram of the complete system is presented in FIG. 10. FIG.10 illustrates how the different parts of the system are implementedtogether.

The block TD-CC represents the center-clipping implementation in timedomain. The block CNG represents the comfort noise generator and it isimplemented in frequency domain after the RER algorithm represented bythe Gr_(1 . . . M−1) Copy block.

While embodiments of the invention have been illustrated and described,it is not intended that these embodiments illustrate and describe allpossible forms of the invention. Rather, the words used in thespecification are words of description rather than limitation, and it isunderstood that various changes may be made without departing from thespirit and scope of the invention.

1. A method of residual echo reduction which follows an acoustic echocancellation (AEC) wherein a microphone signal resulting from anunobservable signal corrupted by an acoustic echo is processed in anattempt to restore the unobservable signal, the acoustic echo being aspeaker signal modified by an acoustic path, wherein an adaptive filtermodels the echo path in the frequency domain, the adaptive filter isapplied to the speaker signal to produce an echo signal that models theacoustic echo, and the echo signal is subtracted from the microphonesignal to produce an output signal that resembles the unobservablesignal but includes a residual echo, the method comprising: receivingthe AEC output signal which still contains the residual echo as a targetsignal; receiving the microphone signal; applying an adaptive residualecho filter to the microphone signal; subtracting the adaptive residualecho filter output signal from the target signal to determine an errorsignal; adapting the adaptive residual echo filter based on the errorsignal; and applying a copy of the adaptive residual echo filter to theAEC output signal to remove any residual echo.
 2. The method of claim 1wherein the microphone signal is combined with the AEC output signal ina weighted average according to a ratio of a near-end component energyand a total microphone signal energy and the adaptive residual echofilter is applied to the weighted average.
 3. The method of claim 1wherein processing takes place independently in a plurality of subbands.4. The method of claim 1 wherein an adaption rate of the adaptiveresidual echo filter is based on a weighted average of a previous gainand a look ahead.
 5. The method of claim 2 wherein the weighting of thelook ahead increases as frequency increases providing greaterflexibility at higher frequencies.
 6. A method of residual echoreduction which follows an acoustic echo cancellation (AEC) wherein amicrophone signal resulting from an unobservable signal corrupted by anacoustic echo is processed in an attempt to restore the unobservablesignal, the acoustic echo being a speaker signal modified by an acousticpath, wherein an adaptive filter models the echo path in the frequencydomain, the adaptive filter is applied to the speaker signal to producean echo signal that models the acoustic echo, and the echo signal issubtracted from the microphone signal to produce an output signal thatresembles the unobservable signal but includes a residual echo, themethod comprising: receiving the AEC output signal which includes theresidual echo as a target signal; receiving the microphone signal;applying an adaptive residual echo filter to the microphone signal;subtracting the adaptive residual echo filter output signal from thetarget signal to determine an error signal; adapting the adaptiveresidual echo filter based on the error signal; and applying a copy ofthe adaptive residual echo filter to the AEC output signal to remove anyresidual echo.
 7. The method of claim 6 further comprising combining themicrophone signal with the AEC output signal in a weighted averageaccording to a ratio of a near-end component energy and a totalmicrophone signal energy and applying the adaptive residual echo filterto the weighted average.
 8. The method of claim 6 wherein processingtakes place independently in a plurality of sub-bands.
 9. The method ofclaim 6 wherein an adaption rate of the adaptive residual echo filter isbased on a weighted average of a previous gain and a look ahead.
 10. Themethod of claim 9 wherein the weighting of the look ahead increases asfrequency increases providing greater flexibility at higher frequencies.